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This paper develops a set of simplified dynamical models with which to explore the conditions 
under which temporal differentiation leads to optimized system output. By temporal differentiation, 
we mean a division of labor whereby different subtasks associated with performing a given task are 
done at different times. The idea is that, by focusing on one particular set of subtasks at a time, 
it is possible to increase the efficiency with which each subtask is performed, thereby allowing 
for faster completion of the overall task. For this paper, we consider a process whereby some 
resource is converted into some final product in a series of three agent-mediated steps. Temporal 
differentiation is incorporated by allowing the agents to oscillate between performing the first two 
steps and performing the last step. We find that temporal differentiation is favored when the 
number of agents is small, and when the process intermediates have a much longer lifetime than the 
original resource. Within the framework of biological systems, we argue that these results provide a 
possible evolutionary basis for the emergence of sleep, and also of distinct REM and non-REM sleep 
states. We also discuss our use of a three-step model. Briefly, in order for temporal differentiation 
to increase product output in a mean-field description of resource metabolism, it is necessary for 
temporal differentiation to have a nonlinear effect on individual process rates. For stochastic models, 
we argue that temporal differentiation can increase product output even in fundamentally linear 
systems. 
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I. INTRODUCTION 

Differentiation and the division of labor is a ubiqui- 
tous phenomenon characterizing the emergence of com- 
plex systems. Different enzymes, nucleic acids and 
other biopolymers are involved in the proper function 
of a living cells. In multicellular organisms, cells dif- 
ferentiate and specialize in the performance of one or 
a few tasks. At higher levels of complexity, multi- 
cellular organisms, e.g. humans, can themselves form 
highly differentiated structures (a modern networked 
economy), where each organism performs one or a few 

tasks aiiisiailiaiiiEm. 

As a result of the ubiquity of the division of labor in 
biology considerable experimental and theoretical work 
has been devoted to understanding both its genetic basis, 
and the selection pressures that give rise to such behav- 
iors. In a general sense, division of labor is favored when 
transport costs associated with delivering process inter- 
mediates to the appropriate agents are small. Therefore, 
division of labor is generally favored at high population 
densities, though this may not always be the case |lfJ.ll4| . 

In this paper, we wish to discuss another form of differ- 
entiation, specifically temporal differentiation. Temporal 
differentiation refers to a division of labor where a given 
task is broken up into several subtasks, and the various 
subtasks are performed at different times. That is, with 
temporal differentiation, a given set of agents performs 
all the subtasks associated with a given task. However, 
these agents concentrate their efforts on one set of sub- 
tasks for a certain period of time, and then concentrate 
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their efforts on another set of subtasks for another period 
of time. This is in contrast to the "standard" picture of 
division of labor, whereby all subtasks associated with a 
given task are performed simultaneously by different sets 
of agents. 

Temporal differentiation, although a perhaps less obvi- 
ous form of division of labor, is nevertheless also a ubiq- 
uitous phenomenon. At the level of task completion by 
humans, it is quite common that various tasks are often 
done in intermittent blocs. Examples include paying of 
bills, housekeeping chores, and the procurement of food. 
Such forms of temporal differentiation are likely prevalent 
in other organisms, since such a temporally differentiated 
labor strategy is a natural approach in many contexts for 
optimizing system efficiency. 

Another, more subtle form of temporal differentiation 
is the phenomenon of sleep. Sleep is prevalent in organ- 
isms with highly complex central nervous systems. It is 
characterized by periods of high levels of alertness, fol- 
lowed by periods during which the brain goes into an 
unconscious state. 

One theory, due to Crick and Mitchison, for the exis- 
tence of sleep, is that sleep is a time when the brain en- 
gages in various garbage collection activities Ell- More 
specifically, sleep is a time when the brain sorts through 
and consolidates information accumulated from the pre- 
vious period of wakefulness El El El El IM EH In- 
creasing evidence suggests that Crick and Mitchison's ba- 
sic hypothesis may be correct j^. Nevertheless, while 
this "garbage collection" hypothesis provides an explana- 
tion for what happens during sleep, it does not explain 
the selection pressures driving the emergence of this phe- 
nomenon. 

In two recent papers [HE1> the author suggested that 
sleep emerges because, in the presence of a day-night cy- 
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cle, it is optimal for a highly complex brain to engage 
in information collection activities during the day, when 
light information is most available, and then to engage in 
information consolidation (i.e. "garbage collection") ac- 
tivities at night, when light information is far less avail- 
able. The idea is that, by concentrating on information 
collection when it is available, and information consol- 
idation when external information is less available, the 
brain can process an optimal amount of information in a 
given amount of time, which presumably confers a sur- 
vival advantage to the organism. 

However, such an explanation is incomplete, since 
some organisms are nocturnal. Further, what is inter- 
esting about the sleep state itslef is that it is divided 
into distinct, alternating cycles of REM and non-REM 
slee p (w here REM stands for "Rapid Eye Movement") 
[THl lid l25j. However, not all sleeping organisms exhibit 
REM and non-REM sleep. In particular, the organism 
Tachyglossus aculeatus, a representative of the earliest 
branch of mammalian evolution (the monotremes), com- 
bines both REM and non-REM sleep into one sleep state 
|26| . This suggests that REM and non-REM sleep are 
not fundamental to sleep itself, but rather emerged via 
the differentiation of a single, older sleep state. 

Therefore, it is possible that temporal differentiation 
of various brain tasks leads to optimized brain function in 
higher organisms, independent of any external day-night 
regulation cycle. The existence of a day-night cycle sim- 
ply regulates the optimal start times for each task period. 
Furthermore, it is also possible that temporal differenti- 
ation is favored the more work needs to be completed 
within a given time period. In the context of REM and 
non-REM sleep, it is possible that mammals with simpler 
brains can perform the information consolidation tasks 
in one sleep state. However, for larger, more complex 
mammalian brains, the amount of information consoli- 
dation to be performed becomes sufficiently large that it 
becomes more efficient to divide the various information 
consolidation subtasks into two distinct sleep states. 

Motivated by these various considerations, we present 
a highly simplified model for the processing of some re- 
source into a final product. The processing of this re- 
source occurs in three separate steps, each mediated by 
an agent. Because our model adopts the notation and 
structure of the theory of chemical reaction kinetics, the 
various agents involved in each subtask are represented 
as enzyme catalysts. 

When there is no temporal differentiation, all three 
subtasks are performed simultaneously, and the frac- 
tion of enzymes assigned to each task is constant in 
time. With temporal differentiation in the context of 
our model, the enzymes oscillate between performing the 
first two tasks for a certain amount of time, and then the 
third task for another amount of time. We use a three- 
subtask model because, in the mean-field approach that 
we adopt here, a minimum of three subtasks is necessary 
in order to achieve the nonlinear dependence of task rate 
on enzyme number required to give an advantage to a 



temporally differentiated labor strategy. For stochastic 
models, this fundamentally nonlinear dependence of task 
rate on enzyme number is not necessary, an issue which 
we will discuss later. 

Within the framework of our model, we find that tem- 
poral differentiation is favored when the amount of avail- 
able agents is at intermediate values compared to the rate 
of resource input, and when the intermediate products 
have a long lifetime compared to the starting resource. 
It therefore makes sense to focus entirely on one set of 
tasks for a time, to convert as much resource as possible 
into a long-lived intermediate, and then switch to pro- 
cessing the intermediate. 

This paper is organized as follows: In the following 
section (Section II), we develop our three-process model, 
and derive the limiting form of the model when the total 
number of enzymes is small. In Section III, we go on 
to solve our model with and without temporal differen- 
tiation. In Section IV, we compare both the temporally 
differentiated and non-differentiated strategies, and de- 
termine the regimes where one strategy is expected to 
be favored over the other. In Section V we discuss our 
results in the context of sleep and other examples of tem- 
poral differentiation in biology. Finally, in Section VI we 
present our main conclusions. We also describe some 
shortcomings of our model, and how our model may be 
further developed in future work. 

II. THE MODEL 

In this section, we introduce our three process model, 
whereby an external resource is converted into a final 
product in a series of three steps. We set up dynamical 
equations governing the production of final product for 
both the temporally differentiated and non-differentiated 
cases. 



A. Definition of the model 

Our model consists of some compartment of fixed vol- 
ume V, into which flows a resource, denoted R\, at some 
fixed rate fn- This resource is processed into a final prod- 
uct, denoted P, via a series of three agent-mediated, or, 
in the language of chemical kinetics, enzyme-catalyzed, 
steps. 

In the first step, an enzyme, denoted E\, binds to 
and then converts R\ into an intermediate i?2- In the 
second step, an enzyme, denoted E2, binds to i?2 7 and 
then converts R2 into an intermediate R3. In the third 
and final step, an enzyme, denoted E3, binds to R3, and 
then converts R3 into the final product P. 

Furthermore, the resource Ri and intermediates R2, 
i?3 have finite lifetimes in the compartment, defined by 
first-order decay constants kn,i, fc_D.2, and kr>,3, respec- 
tively. These decay terms can be due to various fac- 
tors, such as diffusion out of the compartment, or simply 
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the physical decay of the components themselves. In the 
context of networked systems and data processing, these 
decay constants can also correspond to a finite lifetime 
during which an information packet is relevant (say stock 
information that is used by an investor to decide whether 
or not to invest in a given stock). 

In the language of chemical kinetics, the set of reac- 
tions in the compartment is given by, 

Ri — > Decay products (First-order rate constant fczj,i) 
Ei + Ri — > E\ — Ri (Second-order rate constant ku) 
Ei — Ri — > Ei + R 2 (First-order rate constant ki 2 ) 
i?2 — > Decay products (First-order rate constant £0,2) 
E 2 + R 2 — ► E 2 — R 2 (Second-order rate constant k 2 i) 
E 2 — R 2 — > E 2 + R 3 (First-order rate constant fc 2 2) 
R 3 — » Decay products (First-order rate constant £0,3) 
£3 + R3 — > -E3 — -R3 (Second-order rate constant £31) 
E 3 — R 3 ^ E 3 + P (First-order rate constant £32) (1) 

Letting denote the number of particles of resource 
Ri, n Ei the number of particles of enzyme E i: and n EiRi 
the number of particles of Ei — Ri, where i — 1,2,3, we 
obtain the following system of differential equations for 
the time evolution of the numbers of the various compo- 
nents in the compartment: 



can oscillate in time. For the purposes of this paper, 
we will assume that ni, n 2 , and n 3 oscillate in such a 
way that m = rii , n 2 = n 2 , and n 3 = n 3 over some 
time period of length Ti, followed by a time period of 
length T 2 where m = n{ < n\ , n 2 = < n 2 , and 
n 3 = > n 3 . We assume that the total number of 
enzymes remains fixed, however, so that nf + n 2 +n 3 = 

n i + n 2 + n t ■ 

Essentially, if we switch from an enzyme-based view- 
point to an agent-based viewpoint, our model assumes 
that agents can switch from one set of tasks to another. 
In this model, the agents alternate between focusing on 
the first two processes and the third process. In the chem- 
ical kinetics notation, we have, 



Ei/ 2 <-» £3 



(4) 



For the purposes of this paper, we will assume that 
the rate constant of this "task-switching" reaction is 00. 
Therefore, each agent can instantaneously switch from 
one task to another, if it so chooses. Toward the end of 
this paper we will speculate on the effect that a finite 
task- switching rate has on the results presented here. 



C. Limiting forms of the model 



dn Rl fcn 

—j— = Jr - -rrn El n Rl - k D ,in Rl 
at V 

dn El R 1 fcn 

— -j t — = —n El n Rl - k 12 n ElRl 

dn R2 k 2i 

—j— = k 12 n ElRl —n E2 n R2 - k D , 2 n R , 2 

dn E2 n 2 k 2 i 

— -j t — = —n E . 2 n R . 2 - k 22 n E2R2 

dn R3 k 3 i 

—jj- = k 22 n E2li2 - —n E3 n R3 - k D ^n R . 3 

dn E3 n 3 k 3 i 

— -j t — = — n E 3 n R3 - k 32 n E:i R 3 
We also define the quantities ni, n 2 , and n 3 via, 
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We will now study how our model behaves when ni + 
n 2 + n 3 and kn,3 may each be regarded as "small" in 
some sense. The criterion for smallness will be defined 
later, once we have established the behavior of the model 
in these regimes. 

To begin, we note from the previous subsection that 
the various enzyme numbers fluctuate in time. More 
precisely, there exist Ti , T 2 > such that for every inte- 
ger s, the total enzyme numbers for each enzyme are 
at nf, n 2 , respectively, during the time interval 
[s(Ti+T 2 ), s(Ti+T 2 )+Ti], while during the time interval 
[s(Ti + T 2 ) + Ti, (s + l)(Ti + T 2 )], the enzyme numbers 
are at , n 2 , and n 3 . 

If we define, 



ni = n El + n ElRl 
n 2 = n E . 2 + n E2R2 
n 3 = n Es + n E3 n 3 



(3) 



so that m, n 2 , and n 3 denote the total amount of Ei, 
E 2 , and E 3 respectively at some given time. 

Note that our model is essentially a mean-field descrip- 
tion of the dynamics inside the compartment, since we 
are assuming that the amount of each component can 
take on any nonnegative real number. 



and, 



"1/2/3 



A 



n i/2/3 + n i/2/3 



1/2/3 - -+ 



n i/2/3 n i/2/3 



n i/2/3 + n i/2/3 



then it may be readily shown that, 

n i/2/3 = "-1/2/3(1 ± Al/2/3) 



(5) 



(6) 



(7) 



B. Temporal differentiation in the three-enzyme 
compartment model 

Temporal differentiation can occur in our model if we 
do not assume that m, n 2 , and n 3 are fixed, but rather 



For fixed values of Ai , A2 , we wish to develop a form for 
the first four equations assuming that hi, h 2 are small. 
The overall strategy is as follows: Because R 3 is the in- 
termediate that feeds into the third task, our goal is to 
determine the rate of production of R 3 when hi , h 2 are 
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small. By this we mean that we seek to determine, with 
respect to hi and h 2 , the lowest-order term contributing 
to the production rate of R 3 . 

When hi = 0, we have that n ElRl — n El = n R2 = 
n E 2 R2 = n E 2 = 0. If we let riR^o denote n Rl when 
hi = 0, we obtain, 



dt 



Jr - k DA n Rlfl 



(8) 



Differentiating the second equation with respect to m, 
and setting n ElRul = {dn ElRl /dhi)n 1=0 , we obtain, 

dn El R 1 ,i hi ku 

-J t = ~y~\ l ± A i) n fli,o ~ \ — n Rl fi + k 12 )n ElRul 

(9) 

Differentiating the third equation with respect to m, 
and setting n R2i i — (dn R2 / 'dhi)n 1= n 2 =o, we obtain, 



dn. 



dt 



ki2n ElRl s - ko.2 n R 2 ,\ (10) 



III. LONG TIME BEHAVIOR OF THE MODEL 

In the absence of temporal differentiation, the values of 
rii, n 2 , n 3 remain constant, and so we expect the dynam- 
ics to evolve to a steady-state. With temporal differenti- 
ation, the values of ni, n 2 , n 3 oscillate between two sets 
of values over a well-defined time period Ti + There- 
fore, in this situation, we cannot expect the dynamics 
to settle into a steady-state solution. However, we can 
expect the dynamics to settle into a periodic solution. 

In this section, we therefore consider the long-term be- 
havior of our model with and without temporal differen- 
tiation. Without temporal differentiation, that is, assum- 
ing that Ai = A 2 = A 3 = 0, we look for steady-state solu- 
tions to the dynamical system. With temporal differenti- 
ation, we look for periodic solutions to the dynamical sys- 
tem. We will consider the case where Ai = A 2 = A 3 = 1. 
Here, the enzymes devote themselves entirely to the hrst 
two tasks over a time interval of length Ti , and then de- 
vote themselves to the third task over a time interval of 
length T 2 . 



Note that n E2 — n E2R2 = when h 2 = 0. There- 
fore, derivatives of these quantities that only involve 
hi will be when evaluated at (hi,h 2 ) = (0,0). So, 
the lowest-order derivative at n E2R2 that is possibly 
non-vanishing at (hi,h 2 ) = (0,0) is d 2 n E2R2 /(dhidh 2 ). 
Defining n E2R2yl = (d 2 n E2R2 /(dhidh 2 ))(n u n 2 )=(ofi), we 
obtain, 



dn E2R2: i _ k2i_ 
dt V 



{l±X 2 )n R2! i-k 2 2nE 2 R 2> i (11) 



Note that the lowest order term contributing to the 
production rate of i? 3 is given by n E2R2t ihih 2 . 

For the first four equations, we obtain the linearized 
system, 



dn Rlfl 
dt 



= Ir - k D ,in Rl fl 



dn El R 1 ,i ku fcn 

jj— = —(1 ± Ai)n fllj0 - (— n Ru0 + k 12 )n ElRu i 



dn R2 



dt 



ki 2 n ElRu i - k Dt2 n R2t i 



dn E2R2 s _ &21 
dt V 



(I ± \ 2 )n R2 ,i - k 22 n E2R2i i 



(12) 



For the final two equations, we assume that kn,3 = 0, 
giving, 



A. Case 1: Ai 







When Ai = A 2 = A3 = 0, there is no fluctuation in the 
total enzyme numbers for either Ei, E 2 , or E 3 . There- 
fore, the long-time solution is simply a steady-state so- 
lution. Setting the left-hand sides of the four linearized 
equations to gives, 



n El R u i = -j- 
ki2 



fa. 

ki2 



<JL J- k °- 1 

fel2 (fell/V) 



11R2A 



n E2 R 2 .i 



Is, 

ki2 



ko,2 Is- 



fel2 ' (fcll/V) 



fol/V) fcl 2 



k D ,2 k 22 Is, + kD f 

K12 (kn/V) 



(14) 



If we define A and B via, 



, (kn/V) t 

A = — JR 

KD,1 



(15) 



dnR 3 fc 3 i 

—j— = k 22 n E2R2 - —n E3 n R3 

dn EaR3 k 3 i 

— J t — = —n E3 n R3 - k 32 n E3R3 (13) 



We will discuss the assumption of /cd. 3 = later in 
this paper. 



then we obtain that the rate of production of i? 3 is 
ki 2 {k 2 i/V)/k D , 2 {A/B)hih 2 . 

Now, the maximal production rate of P is obtained 
when n R3 = 00. The reason for this is that the rate of the 
third binding step is then infinite, so that the production 
rate of P is only limited by the rate at which the enzyme 
E 3 can convert i? 3 into P. 
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When n Rs =00, the rate of production of P is given by 
^32^£ 3 i?3 = &32 At steady-state, the rate of produc- 
tion of i? 3 must equal the rate of production of P (since 
the production rate of P is equal to the consumption rate 
of -R3 when k E ,a, = 0). Therefore, at steady-state, 



, (k2i/V) ,A _ _ 
ki2—, (-)fiin 2 = fc 32 n 3 

KD,2 O 



(16) 



Now, defining n — n\ + n 2 + 713, a — (ni + n 2 )/n, (3 = 
n\/{ni + n 2 ), we have, at steady-state, 

= fc 12 (4)o»/3(l-/3) n a = Ml-a)n (17) 

at K£),2 -t> 

Therefore, the steady-state production rate of P is 
maximized when (3 = 1/2, and so we wish to solve, 



2 1 1 

a 2 H a = 



7n 



777, 



where, 



7 = 



Kl2 k D , 2 \b]_ 
4fc 32 



(18) 



(19) 



This gives, 

lip -. , /— — — , , 1 + 2 7 n - VI + 47^ 
a = - — [-1+V1 + 47n] =4> 1-a = ■ — — — 



2 771 



27n 



(20) 

The steady-state production rate of P is then given 
by £32(1 — a)n. To distinguish this value of a from the 
value of a we will obtain in the temporally differentiated 
model, we re-denote the a here a un diff- 



B. Case 2: Ai = A 2 = A 3 = 1 

When Ai = A 2 = A3 = 1, then the enzyme levels for 
enyzmes E\ and P2 are at their maximal levels during 
the time interval [s(7i + T 2 ),s(T 1 + T 2 ) + Pi], and are 
not present during the time interval [s(Ti +T 2 )+Ti, (s + 
l)(Ti +T 2 )\. Therefore, in the time interval [s(T l +T 2 ) + 
Pi,(s + l)(Pi + P 2 )], we have n El = n ElRl = n E2 = 
n-E 2 R 2 = 0- 

By periodicity and continuity it follows that, in the 
time interval [s(Ti + P 2 ), s(Pi + T 2 ) + T x ], we have the 
initial conditions = 2ni, n ElRl — 0, riB 2 = 2n 2 , 
n E 2 R 2 = 0- Starting with these initial conditions, we 
have that the solution to the differential equation for 
n Rl ,o is, 

n Rlfi (At) = nRlfi (0)e- k ^ At + -^(l-e- k ^ At ) (21) 

KD,l 

where At = t- s(7\ + T 2 ). 

The periodicity condition means that we want 
rcjii,o(7i + P2) = ^i,o(0), giving n fllj0 (0) = f R /k D>1 , 
which then implies that n Rl fi{At) — f R /kD,i for all At. 



So for Pi, the lowest-order nonvanishing term for n Rl is 
unaffected by a temporal division of labor. 

Turning to ue 1 r 1 ,i, we have, in the time interval 
[s(Ti + T 2 ),s(T 1 + T 2 ) + Pi], the differential equation, 

dn El R u i „feii /ij ,fen f r 7 \ 

t: = 2 -f^i {-rri \- ki2)n ElRl ,i (22) 

at V kd,i V kd,i 

which may be solved to give, 



n ElRul (At) = 2-(l-e- BAt ) 



(23) 



where A and B were defined in the previous subsection. 

The differential equation for n R2i \ takes on two distinct 
forms, depending on the time interval we are in. For 
* G |>(Ti + T 2 ), s(Ti + P 2 ) + Pi], [a(Ti + P 2 ) + Pi, (s + 
l)(Pi + P 2 )], the differential equations are, respectively, 



dn R 2 ,i 01 A 01 A -BAt 1 
— t— = 2ki2— - 2fci2— e -feu,2nfl 2 ,i 
ut i> i> 



fl2,l 



~kD,2 n R 2 



(24) 



These equations may be solved to give, 

n^.i(At) = r^ 2 ,i(0)e- fc ^ At + 2fci 2 ^^(l - e - fc ^ At ) 

K_D.2 



-2fc 



12 



(A/P) _^-BAt _ e -fc D , 2 At-j 



feD,2 - P 



n fla ,i(At)=n JIa> i(ri) 



-fen, 2 (At— Ti) 



(25) 



Since we must have that riR 2l i(Ti +P 2 ) = nfl 2i i(0), we 
have, 

n fl2 ,i(0) - n fl2i i(Pi) e - fc -- 2T2 

= n R2 ^)e- kD ^ Tl+T2) 



+2k 



12 



(A/P) 

( e -k D ,2T 2 _ e -k Di2 {T!+T 2 )^ 



«D,2 



-2fc 



12 



(A/P) 

.^ e -BT x -k Dt2 T 2 _ e -k Di2 {T!+T 2 )^ 



k D ,2 - B 



(26) 



and so, 



n R2tl {0) = 2k 



A 



? — ko, 2 T 2 



12 



Bl- e -k D , 2 (Ti+T 2 ) 



I _ e -feD,2Ti e --BTi _ g-fc D , 2 Ti 



fen, 2 

This gives, 
n fl24 (At) = 2fei 2 ^^ x 



k D ,2 - B 



(27) 



[l-( 



fee, 2 

1 _ e — kn, 2 T 2 



1 _ e -fc D ,2(T 1 +T 2 ) 
fc D , 2 1 - e-^T, + k D , 2 T 2 ) 
k D ,2~B l- e -fe D ,2(P+T 2 ) 
fee, 2 . e -BAti 



Jg-fcD 



fcr>, 2 At 



«D,2 



P 



(28) 
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Finally, the differential equation for ue 2 r 2 i is given fort e [s(7i+T 2 ), s(T 1 +T 2 )+T\], while nE 2 R 2 = 

by, "" forte [s(T 1 +T 2 )+T 1 ,(s + l)(T 1 +T 2 )]. 



driE 2 R 2 ,i k 2 i , 
j f = 2— n R2l i - k 22 n E2 R 2 ,i (29) 



Therefore, for t £ [s(Ti +T 2 ), s(T 1 +T 2 )+T 1 ], we have, 



n E2 R 2 s(At) 



^ 12 k 21 (A/B) x 



&22 &22 6 



-k 22 At 



-( 



+ ( 



1 _ e -k D ,2T 2 
1 _ e -fcD,2(Ti+T 2 ) 
1 _ e -k D , 2 T 2 



ZD.2 



1 



-(BTi+fcB j2 T 2 ) 



-)- 



1 



-fcn 2 At 



fc D ,2-S 1 - e -^,2(T 1+ T 2 ) 'k 22 -k D:2 

k D . 2 1- e -(BTi + k D , 2 T 2 ) k D , 2 (k 22 - k D , 2 ) 



1 _ e -fe D , 2 (T 1+ T 2 ) k D . 2 -B l - e~ k DMTi+T 2 ) 1 (fc D 2 - B)(k 22 - B)\ 



-k 22 At 



k D :. 



-BAt-i 



fcn,2 ~ B k 22 - B 



(30) 



r 



Note then that tie 2 r 2 ,i starts at and then rises to 
a steady-state value. Therefore, in order to maximimize 
the average production rate of R 3 , we must have that 
T\ = oo, so that the average production rate of Rz is 
simply given by the steady-state production rate. 



When ni = n{ 



2m, n 2 



2n 2 , the steady- 



state value of riE 2 R 2 is given by, 



k 12 k 21 (A/B) _ _ 
nE 2 R 2 = 4 — nin 2 (31) 

V 7 KD.2K22 

and so, when Ti is large, the total amount of R3 that is 
produced during the time period T\ is given by, 



Arifl 3 = k 22 riE 2 R 2 Ti = 4 



k 12 k 21 (A/B) 
V k D , 2 



n x n<{r x (32) 



To ensure periodicity of the solution, T 2 must be such 
that the amount of R 3 consumed in the third step is equal 
to the amount of R3 accumulated. Assuming that the 
amount of R3 is infinite at all times (as with the undif- 
ferentiated case, this assumption maximizes the overall 
production rate of P), then the amount of R3 consumed 
is given by 2kz 2 fi3T 2 . We then have, 



h 2 k 21 (A/B) 
V k D , 2 



(2n 1 )(2n 2 )T 1 = k 32 (2n 3 )T 2 



(33) 



Now, note that since all the enzymes are focused on 
either the first two tasks or the third task in the tempo- 
rally differentiated model, we have n = 2n\ + 2n 2 = 2n$. 
Defining = 2n\/n therefore gives, 



k 12 k 21 (A/B) 



V k 



D,2 



0(1 - 0)n 2 T x = k 32 nT 2 



(34) 



Note that the average production rate of P is simply 
given by k 32 nT 2 / '(T 1 +T 2 ). Although T t and T 2 are both 



infinite, 7i/(Ti +T 2 ), T 2 /(T X +T 2 ) are finite, and we can 
determine the optimal split between the two work cycles 
that maximizes the average production rate of P. 

Defining a = T\/(T\ + T 2 ), we get that the average 
production rate of P is simply £32(1 — a)n. Diving both 
sides of the previous equation by Ti + T 2 gives, 



k 12 k 21 (A/B) 



V 



CD, 2 



0(1 - 0)an 2 = k 32 (l-a)n 



(35) 



As with the steady-state solution, note that we can 
maximize the output of P when = 1/2, and so we 
obtain, 



fci 2 fe 2 i (A/B) 
V k Da 



an = 4^32(1 — a) 



Defining 7 as before gives, 
1 



a = 



1 



771 



1 + 7n 



(36) 



(37) 



To distinguish this a from the a defined in the tempo- 
rally undifferentiated case, we re-denote the a defined in 
this subsection by ctdiff- 



IV. COMPARISON OF SYSTEM OUTPUT 
WITH AND WITHOUT TEMPORAL 
DIFFERENTIATION 

We now wish to compare the rate of production of 
P with and without temporal differentiation, to deter- 
mine whether temporal differentiation can optimize sys- 
tem performance. In comparing the production rate of 
P for both the temporally differentiated and undiffer- 
entiated cases, it makes sense to search for long-term 
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solutions that maximize the production of P for both 
cases. For, if we find that temporal differentiation out- 
competes temporal non-differentiation, and if the non- 
differentiated case is operating at a steady-state that is 
not optimal, then we have not proven anything, since it 
is possible that a temporally non-differentiated system 
will produce at least as much P as a temporally differ- 
entiated system, with the appropriate steady-state solu- 
tion. The converse holds if temporal non-differentiation 
outcompctcs temporal differentiation, and temporal dif- 
ferentiation is not running optimally. 

If we expand both the undifferentiated and differen- 
tiated expressions to second-order in 777, we obtain, for 
small 777, that, 



1 - atundiff = 7 n (! - 2 7"0 
for the undifferentiated case, and, 

1 - a d iff = 7n(l — 771) 



(38) 



(39) 



so that temporal differentiation leads to a rate of produc- 
tion of P that is faster than the undifferentiated case. As 
771 increases, the production advantage for temporal dif- 
ferentiation increases over a certain interval. Since both 
values of 1 — a approach 1 as 771 — > 00, the production ad- 
vantage for temporal differentiation reaches a maximum 
and then disappears as 777 grows. 

Now, plugging in the explicit definitions for A and B, 
we obtain that, 



777 : 



ki 2 k 2 i/V kf; 



(40) 



fel2 (kll/V) 



Note then that 777 increases with n and fu, and de- 
creases with ki) t i and ko,2- 



V. DISCUSSION 
A. Justification of model parameter regimes 

Our model assumes that the total number of enzymes 
involved in processing the resource is small, and that the 
intermediate R3 does not decay. The reason for the first 
assumption is that if the number of enzymes involved in 
processing the resource is large, then temporal differenti- 
ation will not lead to an increase in the production rate 
of P. At large enzyme numbers, the rate limiting step to 
the production rate is simply the input rate of external 
resource to the compartment. 

We also assumed that ko,3 = 0, because we claim that 
a low decay rate of the intermediate R3 is a requirement 
for temporal differentiation to result in a higher produc- 
tion rate of P. Intuitively, if the decay rate of R3 is 
significant, then if the enzymes focus on the first two 
processing tasks and not on the third, the unprocessed 
i?3 will simply decay, so that when the enzymes switch 
tasks and focus on processing R3, there will be little left. 



However, when the decay rate of R3 is low, then it 
makes sense for the enzymes to focus on the first two 
processing tasks for a certain time period, for during that 
time the rate of production of R3 is more than double 
what it would be without temporal differentiation. Since 
the i?3 decays very slowly, when the enzymes switch tasks 
and focus on processing R3 into the final product P, lit- 
tle to none of the unprocessed R3 has decayed, so that 
almost all of the R3 produced gets converted into P. 

In this paper, we assumed that fc.o,3 = 0, which means 
that none of the R3 decays. Therefore, optimal produc- 
tion rate of P is achieved with infinitely long cycle times. 
When ko,3 is positive (but still small), the decay of the 
unprocessed R3 means that optimal production rate of 
P requires a finite cycle time. 



B. The small n criterion 

The analytical solution of the first four differential 
equations governing our model explicitly made use of the 
assumption that n\ and 712 are small. We therefore need 
to investigate what the criteria for smallness are. 

When 77 = Hi + 7I2 + 773 is small, the rate of the first 
reaction is given by, 



fn 



-77l 



(41) 



ki2 T (kn/V) 



As 77 1 increases, eventually there will be enough en- 
zyme Ei present to process all of the incoming resource. 
At this point, the reaction rate becomes Jr. Therefore, 
the transition from small fi\ to large n\ behavior is given 
by the crterion [T3 |. 



k^riEiRi = fn 



(42) 



Denoting n\^ rans as the value of Tti where the transition 
from small to large tt-i behavior occurs, we have, 



77l.tr 



CD,1 



fn_ 

k X 2 ^ {kn/V) 



(43) 



Note that the transition point increases as fn and kp,i 
increase, and as k\\ decrease. 

Now, in the small 77 regime, the rate of the second 
reaction is given by, 



k22riE 2 R 



fn 



D ' 2 ~2 + JMUVj 



77l77 2 



(44) 



Setting 77i = 712 = 7712 for maximal reaction rate, and 
solving the equation fc22Ti.E2.R2 = fn f° r ni2> we obtain 



SD.l 



ni2 - trans ~ y (k 21 /v) ^^(fcn/vr 



(45) 



Note that this transition point also increases as fn, ko 1 
and ku.2 increase, and as fen and fci 2 decrease. However, 
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note that because of the presence of the square-root, the 
dependence of nn^trans on these various parameters is 
weaker than in the previous case. 

In any event, the small n expressions that we developed 
in this paper are only valid when either fcr>,i, or ko,2 
are large, or when k\\ or k\2 are small. This general cri- 
terion for small n makes sense: If fn is large compared 
to n, then there are comparatively few agents that can 
handle the incoming flow of resource. If either k^.i or 
/cd,2 is large, then the incoming resource or intermediate 
decays quickly, so that there are insufficient numbers of 
agents that can grab and process the resource or inter- 
mediate before it decays. Finally, if either k\\ or k\2 is 
small, then the agents grab the resource or intermediate 
slowly, and so the resource or intermediate decays before 
the agents have a chance to grab and process them. In 
this case, too, there are insufficient numbers of agents to 
process all of the incoming resource. 



C. When can a temporally differentiated process 
outperform a non-differentiated process? 

We now turn our attention to the problem of when 
a temporally differentiated pathway outperforms a non- 
diffcrcntiated pathway. 

First of all, the advantage of any kind of differenti- 
ation is that, by having agents specialize in one or a 
few subtasks associated with a given task, they can per- 
form this smaller set of subtasks much better than a non- 
specialized agent. Presumably, then, this results in faster 
completion of the whole task 

However, there is a cost to differentiation, due to 
the need to transport intermediates to the appropriate 
specialized agents. When the total number of agents 
is small, the transport costs are sufficiently high that 
the non-differentiated pathway outperforms the differ- 
entiated pathway. At higher agent numbers, however, 
the population density of agents becomes sufficiently 
high that the transport costs become sufficiently low 
that the differentiated path way can outperform the non- 
differentiated pathway |10lll4| . 

The differentiated pathway can only outperform the 
non-differentiated pathway over a finite interval, because 
the production rate of the final product becomes resource 
limited at sufficiently high agent numbers. For these 
agent numbers, the differentiated and non-differentiated 
pathways perform similarly. Therefore, a differentiated 
pathway can only outperform a non-differentiated path- 
way when the agent-to-resource ratio is at intermediate 
values. Furthermore, the greater the flow rate of re- 
source, the greater the likelihood that the differentiated 
pathway will outperform the non-differentiated pathway. 
The reason for this is that, as the flow rate of resource 
grows, the agent number at which the production rate 
of P becomes resource limited is pushed to higher agent 
numbers, and so the interval over which the differentiated 
pathway outperforms the non-differentiated pathway is 



longer. 

Thus far, our discussion has only focused on the non- 
temporal division of labor. With temporal division of 
labor, the agents do not specialize in only one or a few 
subtasks, but rather oscillate between the various sub- 
tasks of the overall process. In principle, given enough 
time, a given agent can "learn" the given subtask and 
optimize its performance to equal that of the special- 
ized agent. If this optimization time is negligible, then 
under certain conditions (such as those in our model), 
the temporally differentiated process will outperform the 
non-temporally differentiated process. 

If the optimization time is positive, a temporally dif- 
ferentiated process can outperform a non-temporally dif- 
ferentiated process if the decay rate of the intermediate 
i?3 is 0. The reason for this is that, for the temporally 
differentiated process, the optimal production rate of P is 
achieved with an infinite cycle time. Therefore, the finite 
optimization time has a negligible effect on the overall 
production rate. 

However, if fc,D.3 > 0, then the temporally differen- 
tiated pathway will only perform optimally with a finite 
cycle time. For if the time period during which the agents 
focus on the first two subtasks is infinite, then the accu- 
mulated i?3 will decay away, resulting in a low production 
rate of P. In this case, the optimization time will have 
an effect on the overall production rate. Here, in order 
for the temporally differentiated pathway to outperform 
the non-temporally differentiated pathway, the produc- 
tion rate of the temporally differentiated pathway with a 
zero optimization time must be sufficiently greater than 
the production rate of the non-temporally differentiated 
pathway, so that the reduction in production rate due to 
a positive optimization time does not erase the superior 
performance of the temporally differentiated pathway. 

Now, when n is sufficiently small, we have already ex- 
plained that a non-differentiated process will outperform 
a differentiated one. For larger n, n may still be suffi- 
ciently small that the low-order expansions used in this 
paper are valid, but large enough that the differentiated 
pathway can overtake the non-differentiated pathway. 

If n is in either of these regimes, then at low values of 
fa, the value of jn may be sufficiently small that the cost 
associated with temporal differentiation means that it is 
not an optimal labor strategy (as measured by comparing 
the values of I — a undif f and I - «<«//). 

At higher values of fa, the value of can rise to a 
level where the temporally differentiated pathway with 
a negligible optimization time is significantly more ef- 
ficient than the non-differentiated strategy. Therefore, 
even with a cost arising from a positive optimization 
time, the temporally differentiated strategy outproduces 
the non-temporally differentiated strategy. 

Thus, in the small n regime, increasing the value of fa 
at a given value of n increases the performance advantage 
of the temporally differentiated labor strategy. 
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D. Implications for sleep 

In the context of sleep, the implications of the results 
presented here are that sleep emerges because the brain 
can process more tasks if it adopts a temporally differ- 
entiated labor strategy. Presumably, the more tasks a 
brain can accomplish within a given amount of time, the 
greater the survival advantage for the organism, provid- 
ing an evolutionary selection pressure for temporally dif- 
ferentiated labor strategies. 

In our model, we have seen that an optimal ratio be- 
tween the times devoted to two different sets of process 
subtasks emerges, even with a constant inflow of external 
resource. In the context of sleep, this suggests a natural 
sleep cycle that can exist independently of any external 
day-night regulation. This also suggests an evolutionary 
basis for sleep that could apply to nocturnal organisms 

M 

This being said, the presence of a day-night cycle could 
nevertheless regulate the exact location of the various 
subtask time intervals. Presumably, it makes sense for 
most organisms to remain alert during the day, when 
external information is most available, and to process 
that information at night, when external information is 
less available (the ability to avoid predators and to hunt 
stealthily are probably the major selection pressures driv- 
ing the emergence of nocturnal organisms) [24| • 

As described in the Introduction, we also argue that 
the model presented in this paper suggests an evolution- 
ary basis for the emergence of distinct REM and non- 
REM sleep states from an earlier undifferentiated sleep 
state. As the brain complexity increases, and the amount 
of information that must be processed during the sleep 
state increases, it becomes more efficient for the brain to 
oscillate between various information processing and con- 
solidation subtasks associated with the sleep state itself 

MM 

VI. CONCLUSIONS AND FUTURE RESEARCH 

This paper presented a highly simplified three-step 
model for the conversion of some external resource into 
a final product. We showed that, when the number of 
agents available for processing the resource is small, and 
when the second intermediate does not decay, then the 
production rate of the final product can be maximized 
if the agents oscillate between the first two subtasks and 
the third subtask. Based on these results, we conjectured 
that sleep and the emergence of REM and non-REM sleep 



are driven by a selective advantage for a temporally dif- 
ferentiated labor strategy. 

Because our model assumed that the second interme- 
diate does not decay, we obtained an optimal cycle time 
that was infinite, though the ratio of times allocated to 
the various sets of subtasks during a given cycle was well- 
defined. For future research, we would like to explore 
how the optimal cycle time is affected when we assume a 
small, but positive value for /cd,3- Presumably, this will 
lead to an optimal cycle time that is finite, for if the cycle 
time is infinite, then when the agents switch to the third 
task, all of the accumulated R3 will have decayed away, 
so that the production rate of P will be 0. 

In the context of biological systems, it is known that 
sleep oscillates between REM and non-REM states with 
a well-defined cycle time. It would be interesting to de- 
velop a model along the lines of the model considered in 
this paper that, based on a few experimentally measur- 
able parameters, could predict the REM/non-REM cycle 
times via an optimization criterion. Along these lines, it 
would be interesting to also develop a model that could 
similarly predict the sleep/ wake cycles of animals that 
have little to no exposure to the sun, such as rats (so that 
their sleep/ wake cycles must be internally regulated) [2^ . 

We explained in the previous section that a finite cy- 
cle time, combined with an optimization time for enzyme 
efficiency, can lead to distinct parameter regimes where 
temporal differentiation either outperforms or underper- 
forms non-temporal differentiation. The characterization 
of these specific parameter regimes is an issue that we 
plan to explore in future work as well. 

Finally, in this paper we considered a fundamentally 
non-linear model. This was necessary, since a linear de- 
pendence of process rates on agent number will not give 
an advantage to temporal differentiation with a mean- 
field description of the dynamics. However, when agent 
numbers are small, stochastic effects can become impor- 
tant, since it is impossible to have fractional agents, and 
since it is often not possible to split tasks (i.e. either a 
task is completed or it is not). In this situation, even a 
fundamentally linear system can optimize its output via 
a temporally differentiated labor strategy. The influence 
of stochastic effects on temporal differentiation is an issue 
that will also be explored in future work. 
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